We study the impact of changes in the competitors' listings in organic search results on the performance of sponsored search advertisements. Using data from an online retailer's keyword advertising campaign, we measure the impact of organic competition on both click-through rate and conversion rate of sponsored search advertisements. We find that an increase in organic competition leads to a decrease in the click performance of sponsored advertisements. However, organic competition helps the conversion performance of sponsored ads and leads to higher revenue. We also find that organic competition has a higher negative effect on click performance than does sponsored competition. Our results inform advertisers on how the presence of organic results influences the performance of their sponsored advertisements. Specifically, we show that organic competition acts as a substitute for clicks, but has a complementary effect on the conversion performance.